Bundle org.nuxeo.elasticsearch.audit

Documentation

README.md
nuxeo-elasticsearch-audit

About

This project provides a backend based on Elasticsearch for Nuxeo for Audit Service.

The idea is to use Lucene / Elastocsearch as storage backend for the Audit trail entries.

Usage of Elasticsearch as a replacement for the JPA based default backend allows to easily make the Audit service scale :
- when there are a lot of entries
  - lot of access
  - custom logging
- when there are a lot of queries
  - reporting
  - usage of sync systems like Nuxeo Drive
- when custom attributes on Audit entries are used
How it works

An Elasticsearch based AuditBackend is contributed at startup to replace the default JPA based one and it use dedicated audit Elasticsearch index to handle storage and queries.

The queries and PageProviders are based on Elasticsearch native DSL : no automatic conversion is done between EJBQL and Elasticsearch DSL.

The orginal Audit service uses a JPA sequence to assign each audit entry a unique id.

In the Elasticsearch implementation, an alternate sequence genaration system is used : nuxeo-elasticsearch-seqgen.

Data Migration

When nuxeo-elasticsearch-audit is deployed it will automatically replace the default JPA implementation.

However, if you have previous data inside JPA you have to migrate it.

The migration is done in a Worker that will simply go through all existing JPA Audit entries to store them inside the Elasticsearch index.

The migration is done using a configurable batch size (default is 1000 entries).

An Automation Operation is provided to trigger the migration Work.

Sample call using curl :
```
curl -H 'Content-Type:application/json' -X POST -d '{"params":{"batchSize":5000}}' -u Administrator:Administrator http://127.0.0.1:8080/nuxeo/api/v1/automation/Audit.StartMigration
```
NB : Migration of 16 Millions entries on a C3.XLarge AWS instance with default PGSQL setup and default embedded Elasticsearch takes about 3h (1500 entries/s).

Building

To build and run the tests, simply start the Maven build:
```
mvn clean install
```
Parent Documentation: README.md
nuxeo-elasticsearch

About

This project aims at providing Nuxeo bundles to integrate Elasticsearch with the Nuxeo Platform.

The idea is to index/query Nuxeo Documents to/from Elasticsearch.

Building

To build and run the tests, simply start the Maven build:
```
mvn clean install
```
Running tests

Using the embedded Elasticsearch

With the default RestClient:
```
mvn -nsu test
# -Dnuxeo.test.elasticsearch.client=RestClient is set by default
```
Or with the TransportClient
```
mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient
```
Using an external Elasticsearch
1. Start an Elasticsearch using docker:
```
docker run -p 9400:9200 -p 9600:9300 docker.elastic.co/elasticsearch/elasticsearch-oss:6.5.3
```
2.a Run the test with RestClient:
```
mvn -nsu test -Dnuxeo.test.elasticsearch.addressList=http://localhost:9400
```
2.b Or run the test with TransportClient:
```
mvn -nsu test -Dnuxeo.test.elasticsearch.client=TransportClient -Dnuxeo.test.elasticsearch.clusterName=docker-cluster -Dnuxeo.test.elasticsearch.addressList=localhost:9600
```
Links
- Administration: http://doc.nuxeo.com/x/UBY5AQ
- Configuring mapping: http://doc.nuxeo.com/x/WxI5AQ
- Overview: http://doc.nuxeo.com/x/iYElAQ
- Open jira tikets

Resolution Order

[182, 787]

The resolution order represents the order in which components have been resolved by the Nuxeo Runtime framework. This range represents the minimal and maximal orders for this bundle's components.
You can influence this order by adding "require" tags in the component declaration, to make sure it is resolved after another component. It will also impact the order in which contributions are registered on their target extension point (see "Registration Order" on contributions).

Components

Maven Artifact

File	nuxeo-elasticsearch-audit-2021.24.3.jar
Group Id	org.nuxeo.elasticsearch
Artifact Id	nuxeo-elasticsearch-audit
Version	2021.24.3

Manifest

Manifest-Version: 1.0
Archiver-Version: Plexus Archiver
Created-By: Apache Maven
Built-By: root
Build-Jdk: 11.0.16
Bundle-ManifestVersion: 1
Bundle-Version: 1.0.0
Bundle-Name: Audit ElasticSearch backend
Bundle-SymbolicName: org.nuxeo.elasticsearch.audit;singleton:=true
Nuxeo-Component: OSGI-INF/elasticsearch-audit-contrib.xml,OSGI-INF/es-
 audit-pageprovider-contrib.xml,OSGI-INF/es-audit-search-contrib.xml

Exports

Json Export Default Json serialization
Json Graph Json dependency graph
Json Contribution Stats Json statistics for contributions
CSV Contribution Stats CSV statistics for contributions

Charts

Raw Data: Json Contribution Stats

Contributions by Code Type

Loading data

Contributions by Target Extension Point

Loading data

Contributions by Studio Source

Loading data